Two-way video systems are available that include a display and camera in each of two locations connected by a communication channel that allows communication of video images and audio between two different sites. Originally, such systems relied on a setup at each site of a video monitor to display a remote scene and a separate video camera, located on or near the edge of the video monitor, to capture a local scene, along with microphones to capture the audio and speakers to present the audio thereby providing a two-way video and audio telecommunication system between two locations. Conventional systems are typically large, fixed in location, have a restricted ability to cope with changes in scene focus, and do not necessarily provide natural communication. Moreover, conventional designs do not take advantage of available, portable display and capture systems such as are found in cellular telephones and similar hand-held devices.
Referring to FIG. 1, there is illustrated a typical prior art conventional two-way video communication system wherein a first user 20a at a first location views a first display device 14a. A first image capture device 10a, which can be a digital camera, captures an image of the first viewer 20a. If the image is a still digital image, it can be stored in a first still image memory 30a for retrieval. A still image retrieved from first still image memory 30a or video 4841-6861-0104.1 images captured directly from the first image capture device 10a will then be transmitted using a first communication channel 60 to a second display 14b where a second user 20b can view the captured image(s) on a display 14b. Similarly, a second image capture device 10b, which can be a digital camera, captures an image of second viewer 20b at a second location remote from the first location. The captured image data is transmitted through a second communication channel 62 to the first display 14a for viewing by first viewer 20a. The captured image data can also be stored in a second still image memory 30b for retrieval. The first communication channel 60 may be the same as the second communication channel 62.
Although such two-way video communication systems have been produced and used for high-end teleconferencing and other two-way video communications applications, there are some significant practical drawbacks for incorporation in a portable and hand-held video communication systems. Expanding the usability and quality of such portable video communication systems has been the focus of much recent research, with a number of proposed solutions directed at making it easier to provide two-way video communication. However, numerous problems continue to exist that are peculiar to the hand-held environment, such as image quality, ease of use, security, and privacy, that have not yet been adequately addressed.
One relevant problem for hand-held communication devices, are their inability to compensate for observer motion or vibration, poor image capture angles, and changes in the field of view.
U.S. Pat. No. 7,058,433 entitled “Mechanism for Ergonomic Integration of a Digital Camera into a Mobile Phone” to Carpenter, discloses a pivot provided between the display and the camera sections of an imaging device so that the camera can be better set up for capture during video communication. U.S. Patent Application Publication 2006/0148517, entitled “Video Conference Feature for a Mobile Communication Terminal” by Yu, discloses a display is that can be tipped up and the image inverted for better viewing during video communication. However, neither Carpenter nor Yu address the problem of poor capture angle produced by the low hand position (see FIG. 12) that is commonly used by people when looking at the handheld display of a mobile phone camera or other portable imaging and display device.
U.S. Patent Application Publication No. 2004/0070666 entitled “Method and Apparatus for Transmitting a Video Image”, by Bober, discloses use of face detection for locating the first participant's face in the captured image and cropping the captured image prior to transmission to the second participant to help keep the face centered in the image as viewed by the second participant. This approach does not address poor capture angle or image shake produced by hand jitter as is commonly encountered when the portable imaging and display device is operated with one hand while the user is in motion. Additionally the method disclosed by Bober does not suggest alteration of the background to improve privacy or reduce data transmission rate.
Privacy issues are particularly relevant for a portable imaging device as it is being transported from one environment to another. U.S. Patent Application No. 2004/0257431 entitled “Video Conferencing Apparatus and Method”, Girish discloses an iris shaped cover for the imaging device to provide visual confirmation that the video capture capability is disabled. Similarly, Girish discloses a light to indicate that video capture and audio capture are enabled. However, this approach does not provide a sufficiently versatile approach wherein the user would like to show his face during video communication without showing the background from the environment or would like to select a different background that is more suitable for the recipient of the image.
The combination of image capture, image processing and wireless communication capabilities in a hand-held communication device such as a cellular phone with an embedded camera make it uniquely suited for use as a portable security system based on imaging and audio. Face recognition algorithms based on image comparison with sophisticated image processing systems is taught by P. Viola, M. J. Jones, “Robust Real-Time Face Detection,” International Journal of Computer Vision, Vol. 57, No. 2, May 2004, pp. 137-154. A good article describing face recognition techniques for video imaging is contained in the article by G. Aggarwal, A. Chowdhury, R. Chellappa, “A System Identification Approach for Video-Based Face Recognition”, Proc. of the International Conference on Pattern Recognition, 23-26 Aug. 2004, Cambridge, UK.
Further, the use of biometric information in a dedicated security system is well known in the art as well, U.S. Pat. No. 6,853,739 to Kyle discloses a system that is based on the use of captured biometric information (face, fingerprint, hand, iris) that is compared against a database of many individual sets of biometric information, including face recognition, to verify the identity of an user from a group of individuals. The system as disclosed by Kyle utilizes a remote central processing unit on a computer network to perform the data analysis for comparison of the captured biometric information to the many individual sets of biometric information from the group of individuals. Thus, the system as described by Kyle is a dedicated security system where the biometric capture device must be setup to communicate with a central processing unit wherein the captured biometric information is compared to the database with an algorithm to make a decision regarding the identity of the user.
Similarly, Akiyama in U.S. Pat. No. 6,724,919, discloses a method for a system that uses a face identification algorithm on a computer system to verify the identity of the user and if a new user is detected, then a preset process is executed to change the computer setup to that desired by the new user. Thereby, Akiyama teaches the use of a face identification system to improve the operation of the computer alone and does not anticipate a portable personal security system.
While both Kyle and Akiyama disclose useful techniques for verification of the identity of a user, neither Kyle nor Akiyama teach simplified techniques that would be implementable in a hand-held communication device. In addition, Kyle and Akiyama do not teach the use of a portable imaging and display device as a versatile, self-contained personal security system.
Portable imaging and display devices or systems require special solutions due to their small size, limited capabilities, inherent problems, and high mobility. Conventional solutions do not address the problems of vibration and capture angle induced by one-handed operation of a portable imaging device nor do these solutions adequately address the problems of privacy, background suitability, or security for effective communication. In addition, a portable personal security system wherein the user identification can be verified and the identification information can be wirelessly communicated to other devices is not disclosed.